Welcome to our October 2023 monthly digest, where you can catch up with any AIhub stories you may have missed, peruse the latest news, find out about recent events, and more. This month, we talk AI, bias, and ethics with Aylin Caliskan, learn more about probabilistic logic shields, knowledge bases, and sparse reward tasks, and find out why everyone should learn a little programming. AIhub ambassador Andrea Rafai met with Aylin Caliskan at this year's International Joint Conference on Artificial Intelligence (IJCAI 2023), where she was giving an IJCAI Early Career Spotlight talk, and asked her about her work on AI, bias, and ethics. In this interview they discuss topics including bias in generative AI tools and the associated research and societal challenges. In their IJCAI article, Safe Reinforcement Learning via Probabilistic Logic Shields, which won a distinguished paper award at the conference, Wen-Chi Yang, Giuseppe Marra, Gavin Rens and Luc De Raedt provide a framework to represent, quantify, and evaluate safety.

Image created by author using DALL.E. Are you excited about self-driving cars? Would you trust autonomous driving technology if you were invited to step into a self-driving vehicle? Most importantly, how do you know whether you would be safe during the journey? Safety is difficult to measure, unlike accuracy, which we are more familiar with.

Safe Reinforcement learning (Safe RL) aims at learning optimal policies while staying safe. A popular solution to Safe RL is shielding, which uses a logical safety specification to prevent an RL agent from taking unsafe actions. However, traditional shielding techniques are difficult to integrate with continuous, end-to-end deep RL methods. To this end, we introduce Probabilistic Logic Policy Gradient (PLPG). PLPG is a model-based Safe RL technique that uses probabilistic logic programming to model logical safety constraints as differentiable functions. Therefore, PLPG can be seamlessly applied to any policy gradient algorithm while still providing the same convergence guarantees. In our experiments, we show that PLPG learns safer and more rewarding policies compared to other state-of-the-art shielding techniques.